TinyGAN: Distilling BigGAN for Conditional Image Generation

Generative Adversarial Networks (GANs) have become a powerful approach for generative image modeling. However, GANs are notorious their training instability, especially on large-scale, complex datasets. While the recent work of BigGAN has significantly improved quality generation ImageNet, it requires huge model, making hard to deploy resource-constrained devices. To reduce model size, we propose black-box knowledge distillation framework compressing GANs, which highlights stable and efficient process. Given as teacher network, manage train much smaller student network mimic its functionality, achieving competitive performance Inception FID scores with generator having \(16\times \) fewer parameters. (The source code trained publicly available at https://github.com/terarachang/ACCV_TinyGAN).